Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adaptive Zero Determinant Strategy #1282

Open
wants to merge 14 commits into
base: master
Choose a base branch
from

Conversation

dashiellfryer
Copy link
Contributor

My recent graduate Emmanuel Estrada and I made a class that adapts its four vector in response to payoff. The player maintains zero determinant bounds each round. In our tests this player eventually outperforms all the standard zero determinant strategies. We tested by using 2000 rounds instead of the standard 200. We look forward to any comments.

Copy link
Member

@drvinceknight drvinceknight left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks a lot for contributing this!

The tests are not passing but I believe this is not due to your code contribution but to an upgrade of pandas which I've captured here #1283. I don't suggest you fix that here as it involves the result set which can be a bit of a pain so for now could you modify one line in the requirements.txt file. Line 7 currently reads:

pandas>=0.18.1

Could you change that to be:

pandas==0.25.3

This will just make sure the test suite runs correctly and we'll fix the pandas issue elsewhere.

I haven't looked at the logic itself yet but we are going to need some automated tests for the strategy as well. Is this something you would like a hand with?

I've also left some very minor requests for formatting/style changes.

Thanks again for this contribution, I look forward to getting it in :)

axelrod/strategies/adaptivezerodet.py Outdated Show resolved Hide resolved
axelrod/strategies/adaptivezerodet.py Outdated Show resolved Hide resolved
axelrod/strategies/adaptivezerodet.py Outdated Show resolved Hide resolved
axelrod/strategies/adaptivezerodet.py Outdated Show resolved Hide resolved
Copy link
Member

@drvinceknight drvinceknight left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is looking good, a few requests in terms of writing the tests and adding some documentation. Happy to help with that.

"manipulates_source": False,
"manipulates_state": False,
}

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could we add few tests for the two specific methods: _adjust_parameters and score_last_round (just running the methods and checking that the attributes have changed accordingly?

Comment on lines +342 to +345
player = self.player(l=R)
axelrod.seed(0)
match = axelrod.Match((player, axelrod.Alternator()), turns=200)
match.play()
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This tests need to be tweaked so that we can check expected behaviours. There's information on writing tests here: https://axelrod.readthedocs.io/en/stable/tutorials/contributing/strategy/writing_test_for_the_new_strategy.html

For example this specific paragraph would be something like (although I am not sure what the expected behaviour is):

Suggested change
player = self.player(l=R)
axelrod.seed(0)
match = axelrod.Match((player, axelrod.Alternator()), turns=200)
match.play()
expected_actions = [(C,D), (D, C)] * 100
self.versus_test(opponent=axelrod.Alternator),
expected_actions=expected_actions, seed=0, init_kwargs={"l": R})

Note that we could also check that the other attributes have changed accordingly using the attrs keyword.

Let me know if I can assist with that (happy to PR the tests to your branch).

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree but I didn't want to search for seeds just to have to do so again once #1288 goes in. The current tests make sure that at least each branch is run (so that the limits are bounced into).

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I understand. Are you suggesting this are place holders and we merge #1288 and then fix these?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, after we merge this I'll update all the tests that require seeds in #1288 (there are many and most will have to change).

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sounds good. Happy to take a share of that work if helpful.


class AdaptiveZeroDet(LRPlayer):
"""A strategy that uses a zero determinant structure that updates
its parameters after each round of play.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would be helpful to have some more information here, perhaps an explanation of the update procedure? It's ok to go quite in depth and include some mathematics which we can then use to check that the _adjust_parameters method is working as expected.

@marcharper
Copy link
Member

Hi all, quick update on this one. There are really four possibly strategies here depending on how one wants to update the underlying parameters each round. It's also not clear how much to update each parameter, so I'd like to run some evolutionary algorithms to try to fit these hyperparameters.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants